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INTRODUCTION 

Intensive  care  unit  clinicians  may  be  confronted  by  more  than  200  variables  for  critically-ill  patients  during 
rounds  each  morning. (1)  In  contrast  people  are  not  able  to  judge  the  degree  of  relatedness  between  more 
than  two  variables  without  assistance.(2)  A  late-2009  report  by  the  National  Research  Council  (NRC)  suggests 
that  information  technology  (IT)  solutions  for  medicine  should  shift  their  focus  from  current  solutions  that 
provide  cognitive  support  on  each  individual  piece  of  equipment  or  report  to  one  that  supports  clinicians  in 
synthesizing  massive  amounts  of  data  to  acquire  a  conceptual  model  of  the  whole  patient.  (3)  The  hypothesis 
of  this  research  effort  is  to  demonstrate  that  reducing  clinician  cognitive  load  by  consolidating  complex 
multimodal  physiological  information  into  coherent  images  using  data  visualization  will  result  in  reductions  in 
medical  error  and  improvements  in  patient  care,  safety,  and  efficiency.  Our  research  objective  is  to  design, 
develop,  and  test  a  prototype  data  visualization  system  that  integrates  and  presents  complex  disparate  data 
streams  into  dynamic  visualizations  that  enable  clinicians  to  more  easily  assimilate  the  information  and  derive 
insight  from  it  resulting  in  faster  more  accurate  treatment  decisions.  There  are  4  phases  to  this  research. 

Phase  1  is  to  develop  a  paper  prototype  interactive  visualization.  In  Phase  2  and  3  Columbia  University 
investigators  focus  on  data  acquisition  and  analysis  of  data  to  construct  patient  cases  for  the  purpose  of  the 
visualization  experiments  while  Draper  Laboratory  investigators  focus  on  developing  the  visualization  tools 
detailed  in  Phase  1 .  During  Phase  4  a  set  of  experiments  will  be  conducted  to  evaluate  the  effectiveness  of 
the  visualization  against  an  unaided  standard-of-care  condition. 


KEYWORDS 


KEYWORDS:  data  visualization;  data  reduction;  linear  projections;  clinical  decision  making;  intensive  care; 
physiological  data;  sepsis 

ACCOMPLISHMENTS 

What  were  the  major  goals  of  the  project? 

Milestone  1:  Prototype  visualization  development  complete 
Duration:  3  months  (Month  1-3) 

Percentage  of  Completion:  100% 

Milestone  2:  Data  collection,  analysis,  and  deployment  to  back-end  data  system  complete 
Duration:  6  months  (Month  4-9) 

Percentage  of  Completion:  100% 

Milestone  3:  Interactive  visual  interface  complete  and  connected  with  the  data 
Duration:  6  months  (Month  4-9) 

Percentage  of  Completion:  100% 

Milestone  4:  Conduct  Cognitive  Experiment  to  evaluate  visualization  tool 
Duration:  3  months  (Month  9-12) 

Percentage  of  Completion:  100% 

What  was  accomplished  under  these  goals? 

The  Statement  of  Work  on  this  project  is  for  Columbia  University  and  Draper  Laboratory  Investigators  to 
develop  novel  visualization  tools  that  represent  health  data  acquired  from  patients  in  the  neurological  intensive 
care  unit  of  Columbia  University  Medical  Center.  The  general  aim  of  this  effort  is  to  demonstrate  that 
interactive  data  visualizations  designed  to  transform  and  consolidate  complex  multimodal  physiological  data 
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into  integrated  interactive  displays  will  reduce  clinician  cognitive  load  and  will  result  in  reductions  in  medical 
error  and  improvements  in  patient  care,  safety,  and  efficiency.  Three  case  scenarios  to  focus  on  were 
identified  in  the  scope  of  work.  The  first  focus  area  is  to  develop  a  visualization  to  help  clinicians  identify 
patients  that  are  changing  physiological  states  and  therefore  require  medical  attention.  We  proposed  applying 
empirical  orthogonal  function  (EOF)  analysis  to  identify  patient  states  and  variables  that  have  changed  over  a 
specified  time  period.  Once  the  clinician  is  alerted  to  a  change  in  patient  state,  decision  support  tools  should 
facilitate  their  clinical  investigation  and  in  making  a  specific  decision  that  fall  into  one  of  three  general 
categories:  1)  identify  a  physiologic  end  point  to  guide  goal-directed  therapy,  and  determine  the  relative 
effectiveness  of  a  specific  or  set  of  interventions  in  achieving  those  goals,  2)  determine  the  onset  of  a 
secondary  complication  and  intervene,  3)  identify  active  physiologic  processes  that  can  affect  overall  clinical 
management.  The  project  is  broken  down  into  four  key  deliverable  milestones  including:  1)  prototype 
visualization  development,  2)  data  collection,  analysis,  and  deployment  to  back-end  data  system,  3)  interactive 
visual  interface  complete  and  connected  with  the  data,  and  4)  conduct  cognitive  experiments  to  evaluate  the 
effectiveness  of  the  visualization  tools  against  an  unaided  condition. 

Draper  Laboratory  human-factor  engineers  gained  a  comprehensive  understanding  of  the  needs  and  data 
requirements  for  the  three  case  scenarios  by  completing  (3/15/13)  a  cognitive  work  analysis  (CWA)  with 
clinical  staff  from  the  Columbia  University  Neurological  Intensive  Care  Unit.  Columbia  University  Investigators 
and  Draper  Laboratory  human-factor  engineers  underwent  an  iterative  process  to  translate  the  CWA  results 
into  requirements  for  prototype  visualizations.  Mockup  visualizations  were  developed  for  an  integrated  system- 
level  overview  that  utilizes  empirical  orthogonal  functions  (complete  4/26/2013),  and  for  specific  decisions 
related  to  patient  development  of  sepsis  (complete  4/15/2013)  and  intracranial  pressure  management 
(complete  4/26/2013).  We  deployed  a  back-end  MongoDB  database  and  have  been  able  to  deposit  data  into 
it.  However,  we  found  that  building  an  interface  to  connecting  the  visualization  tool  directly  to  this  data  source 
was  not  realistic  given  the  time  constraints  and  need  for  experimenter  control  in  the  experiment.  Instead  we 
created  static  data  files  that  are  read  directly  for  the  purpose  of  the  experiment.  As  we  stated  in  our  quarter 
report  this  has  not  impacted  achieving  the  primary  goals  of  the  project.  Alternatively  however  we  have  been 
able  to  incorporate  elements  of  our  visualization  tools  into  a  web-based  visualization  tool  to  connect  to  a  real¬ 
time  data  stream  (Milestone  2)  as  will  be  detailed  later  in  this  report.  The  results  of  the  cognitive  work  analysis 
(reportable  outcome  1)  and  prototype  mockups  (reportable  outcome  2)  are  detailed  below.  Each  mockup 
underwent  a  series  of  design  iterations  to  ensure  that  the  visualizations  made  sense  for  the  tasks  and 
decisions  that  the  clinicians  need  to  solve. 

Patient  Overview  Visualization 

The  goal  of  the  patient  overview  visualization  is  to  identify  patient  state  changes  and  identifying  patients  that 
need  clinician  attention.  The  most  difficult  task  for  clinicians  is  not  determining  which  patients  are  doing  poorly 
-  this  is  usually  self-evident  -  it  is  determining  which  patients  are  changing  and  require  intervention.  We 
hypothesize  that  an  interactive  visualization  can  be  designed  to  support  clinicians  in  more  quickly  recognizing 
changes  to  patient  state,  situation  awareness,  and  faster  ability  to  pinpoint  emerging  situations.  Empirical 
orthogonal  function  analysis  is  a  multivariate  temporal  analysis  technique  that  can  be  used  to  identify  sources 
of  variance  in  patient  data  and  what  physiological  parameters  vary  together. 

During  the  second  quarter  we  set  out  to  determine  the  best  way  to  present  empirical  orthogonal  function  (EOF) 
results  to  clinicians  to  denote  patient  state  changes  and  identify  variables  accounting  for  the  variance.  Data 
preprocessing  was  an  essential  first  step  to  applying  this  technique  and  we  addressed  two  characteristics  of 
the  data,  specially  outlier  removal  and  missing  data.  As  a  result  of  the  fact  that  the  physiological  data 
collected  in  the  Neurological  ICU  is  not  collected  in  a  controlled  environment,  there  are  frequent  outliers  that 
represent  spurious  data  from  the  perspective  of  patient  care. 
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-  For  example,  catheters 

Figure  1.  Unfiltered  (top)  versus  filtered  (bottom)  temperature  values  measuring  body  temperature  are 

often  removed  prior  to  the 
termination  of  data  collection 
leading  to  temperature  readings 
in  both  the  body  temperature 
range  as  well  as  temperature 
readings  in  the  room 
temperature  range.  We 
confronted  difficulties  leveraging 
existing  computational  tools 
given  the  high  levels  of 
missingness  in  our  datasets. 

We  explored  interpolation  and 
matrix  factorization  methods  for 
imputing  missing  values  in  the 
multidimensional  setting.  We 
also  explored  custom  extensions 
to  existing  matrix  factorization 
methods  to  adapt  them  to  time 
series  and  potentially  identify 
spurious  outliers  simultaneously.  The  best  procedure  we  found  was  to  implement  a  variational  Bayesian 
mixture  model  for  the  detection  and  removal  of  such  spurious  states  -  the  results  of  which  applied  to  the  same 
variable  (temperature)  can  be  seen  in  the  figure  1 . 
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Figure  2.  EOF  Analysis  (top)  and  temperature  (TMP-2;  bottom) 
values 
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Empirical  orthogonal  function  (EOF) 
analysis  is  designed  to  identify  and 
highlight  the  set  of  variables  that  are 
contributing  most  to  the  variability  of  a 
patient's  state  during  a  given  time 
period.  (4,  5)  This  analysis  can  be 
used  both  for  visualizing  the  patient's 
state  over  time,  but  can  also  be  used  to 
identify  points  of  significant  change. 

Practically,  due  to  the  fact  that  many 
physiological  variables  are  not  always 
recorded,  we  had  to  first  address  the 
issue  of  missingness.  Although,  we 
experimented  with  several  approaches, 
the  combination  of  outlier  removal  as 
described  above  and  constant 
interpolation  was  determined  to  be  the 
most  accurate  approach.  We  have 
performed  the  EOF  analysis  on  2  fully 
anonymized  patient  datasets.  A  brief 

period  from  one  of  these  patients  is  displayed  in  figure  2  highlighting  a  period  of  significant  change  where 
temperature  is  one  of  the  variables  involved  in  this  change. 
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Figure  3.  EOF  Analysis  Results  for  Individual  Patient 
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Several  steps  are  required  to  apply 
EOF  to  physiologic  data.  The  raw 
data  must  be  visualized  and 
preprocessed  on  a  per-variable 
basis  with  variational  Gaussian 
mixture  models  for  outlier  removal. 
Interpolation  is  then  used  to  create 
a  uniformly  sampled  time  series 
based  on  the  original  series.  Data 
is  separated  across  all  variables 
into  time  bins  and  for  each  bin  the 
primary  eigenvector  is  calculated. 
This  represents  the  unitized  linear 
combination  of  variables  that 
explains  the  most  variance  for  the 
bin.  Each  patient’s  primary 
eigenvector  for  each  time  bin  can 
be  visualized  on  a  heatmap  (See 
Figure  3.)  We  quickly  discovered 
that  while  heatmaps  accurately 
represented  the  varying 
relationships  among  all  the 


parameters  that  it  would  be  difficult  to  use  this  information  to  determine  if  the  patient  was  changing  in  ways  that 
required  attention.  This  is  a  key  point  of  learning  and  emphasis,  specifically  that  visualizing  relationships 
among  patient  data  parameters  accurately  does  not  in  itself  solve  the  real  problem  of  helping  clinicians  make 
better  decisions  from  patient  data.  At  best  this  heatmap  requires  significant  effort  on  part  of  the  users  to 
extract  meaningful  information  from  it.  Data  visualization  in  itself  then  is  not  sufficient  and  we  thereafter  added 
the  requirement  that  an  analysis  must  both  represent  the  relationships  among  parameters  and  convert  this 
information  into  a  form  that  fits  into  the  mental  framework  of  clinicians. 


In  the  second  and  third  quarter  we  worked  to  determine  the  best  way  to  present  results  to  clinicians  to  denote 
patient  state  change  and  to  highlight  what  variables  account  for  this  variance.  We  expanded  our  approach  to 
include  an  alternative  method  of  applying  EOF  analysis.  In  contrast  to  the  method  that  we  previously 
described,  this  method  evaluated  the  directions  in  the  space  of  measured  variables  that  explains  the  most 
variance  over  all  time  for  all  See  Figure  4  for  an  example  principal  EOF  weighting  along  with  the  composite 
score  for  patient  2234.  The  principal  EOF  seems  to  capture  the  sympathetic  response  with  an  increase  in 
heart  rate  and  blood  pressure.  This  can  be  seen  in  Figure  4  from  minute  200  to  minute  450.  During  this  time 
period  rises  in  both  blood  pressure  and  heart  rate  lead  to  a  rise  in  the  principal  EOF. 

This  method  can  be  extended.  Instead  of  allowing  the  statistics  of  the  data  to  determine  the  most  meaningful 
weightings  as  described  above,  it  is  also  possible  to  choose  what  should  be  an  a  priori  meaningful  weighting. 
We  focused  on  cardiovascular  collapse  as  well  as  respiratory  distress.  We  leverage  clinical  knowledge  to 
identify  cardiovascular  collapse  as  a  scenario  where  blood  pressure  is  falling  and  other  vitals  signs  such  as 
heart  rate  and  respiratory  rate  are  rising.  For  example  it  is  well  known  that  patients  headed  for  septic  shock 
start  to  have  drops  in  blood  pressure  while  heart  rate  increases.  In  practice  these  subtle  changes  over  hours 
can  be  difficult  to  pick  up  by  clinicians  because  heart  rate  and  blood  pressure  are  typically  in  the  normal  ranges 
until  the  patient  is  on  the  verge  of  shock.  For  example  Figure  5  (top)  demonstrates  that  at  approximately 
minute  120,  the  patient's  heart  rate  rises  while  blood  pressure  rises  moderately,  followed  by  a  stabilization  of 
heart  rate  and  a  decline  in  blood  pressure.  During  both  of  these  phases,  the  CV  index  rises.  Also,  for 
respiratory  failure,  we  identify  that  scenario  by  a  decreasing  oxygen  saturation,  decreasing  respiratory  rate, 
and  increasing  heart  rate.  Figure  5  (bottom)  demonstrates  that  from  approximately  minute  2000  to  minute 
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Figure  4.  EOF  Composite  Score  for  Individual  Patient  4000,  patient  2247  experienced  increasing 

respiratory  distress  marked  by  a  gradually 
increasing  respiratory  rate  and  a  gradually 
decreasing  oxygen  saturation.  The  RESP 
Index  captures  these  changes  with  a  rising 
value  over  this  time  period. 


We  believe  there  are  several  benefits  to  this 
approach  including:  1)  it  captures  clinically 
relevant  changes  across  physiologic 
parameters;  2)  it  may  be  generalized  to  any 
number  of  parameters  and  patterns  deemed 
important  to  monitor  for;  3)  it  presents  the 
results  to  clinicians  as  an  index  that  can  be 
tracked  over  time  fitting 
patients.  The  initial  method  we  outlined 
evaluates  a  similar  the  existing  cognitive 
model  in  intensive  care;  and  4)  results  can 
also  be  displayed  in  an  overview  format 
From  top  to  bottom:  The  variable  weightings  for  the  first  EOF,  showing  many  patients  at  once.  During  the 

the  projection  of  the  first  EOF  onto  patient  2234's  multivariate  cognitive  experiment  we  will  test  these 

time  series,  patient  2234's  heart  rate  time  series  and  patient  hypotheses  and  evaluate  the  perceived 

2234's  blood  pressure  time  series. _  helpfulness  of  these  indices  to  clinicians  to 

identify  patient  change  that  may  precede  clinical  deterioration.  Figure  6  shows  the  iterations  of  the  overview 
page  to  help  clinicians  identify  which  patients  need  attention  first.  However,  any  results  would  need  to  be 
tempered  until  clinical  studies  can  be  conducted  to  determine  if  specific  indices  are  associated  with  clinical 
conditions  and  to  determine  values  for  indices  that  may  represent  different  categories  of  clinical  risk  that  should 
be  monitored  for. 
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Figure  5.  EOF  Composite  Score  for  Cardiovascular  Collapse 
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A  pre-specified  weighting  corresponding  to  cardiovascular  collapse  that  is 
demonstrated  during  a  portion  of  patient  2229's  time  series.  From  top  to  bottom: 
The  cardiovascular  collapse  variable  weightings,  the  projection  of  the  weightings 
onto  the  multivariate  patient  time  series  to  create  the  CV  Index,  patient  2229's 
heart  rate  time  series,  and  patient  2229's  blood  pressure  time  series. 
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Figure  5b.  A  pre-specified  weighting  corresponding  to  respiratory  distress  that  is 
demonstrated  during  a  portion  of  patient  2247's  time  series.  From  top  to  bottom: 
The  respiratory  distress  variable  weightings,  the  projection  of  the  weightings  onto 
the  multivariate  patient  time  series  to  create  the  Resp  Index,  patient  2247's 
respiratory  rate  time  series,  and  patient  2247's  oxygen  saturation  time  series. 
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Figure  6.  EOF  Composite  Score  Overview  Format 
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Figure  6.  The  patient  overview  page  provides  clinicians  1  hour,  12  hour,  and  24  hour  windows  for  change  on 
two  pre-specified  and  two  unspecified  EOF  clinical  indices.  We  will  test  in  a  cognitive  experiment  whether 
clinicians  are  able  to  identify  patients  that  need  attention  first  using  this  approach. 


Sepsis  Visualization 

The  goal  of  the  SIRS  (Sepsis)  visualization  is  to  help  clinicians  identify  the  physiological  signs  leading  to 
sepsis.  This  is  challenging  for  clinicians  because  with  sepsis  there  are  a  series  of  5-6  variables,  sometimes 
more,  (HR,  HRV,  Respiration  Rate,  Temp,  WBC  (and  bands),  BP)  that  can  be  used  as  indicators  -  perhaps 
early  indicators  -  of  a  patient  going  into  septic  shock.  Generally,  if  two  or  more  criteria  are  met,  there  is  a 
suspected  infection. 

1 )  temperature  >38QC  or  lower  than  36eC; 

2)  heart  rate  >  90  beats/min; 

3)  respiratory  rate  >  20  breaths/min;  and 

4)  white  blood  cell  count  >  12,000/mm3  or  <  4,000/mm3  which  more  than  10%  immature  forms. 

At  Columbia,  clinicians  monitor  6  variables  (4  SIRS  criteria,  and  2  additional): 

1 )  temperature 

2)  heart  rate 

3)  respiratory  rate 

4)  white  blood  cell  count 

5)  heart  rate  variability 

6)  blood  pressure 

The  challenge  is  that  once  these  criteria  are  met,  it  is  often  the  case  that  the  patient  is  severely  ill.  It  is  possible 
that  we  can  help  to  detect  the  early  emergence  of  sepsis  through  supporting  clinicians  in  seeing  early  trends  in 
SIRS-related  variables.  The  most  difficult  part  (for  clinicians)  of  detecting  (early  detection)  that  a  patient  is 
potentially  going  into  septic  shock  is  in  seeing  the  slight  trending  of  multiple  variables  at  one  time.  Not  a  single 
variable  spiking,  but  instead,  it  is  the  trend  of  all  variables.  The  early  trend  of  these  variables  that  we  want  to 
support  is  that  temp,  HR,  RR,  and  WBC  go  up  and  HRV  and  BP  go  down.  Typically,  these  variables  change 
close  to  the  same  time. 


to 


For  clinicians,  the  challenge  is  the  monitoring  for  these  slight  trends  in  multiple  variables  are  multifaceted: 

1 )  Monitoring  for  the  emergence  semi-rare  events  with  weak  evidence  is  challenging. 

2)  The  variables  that  the  clinicians  are  monitoring  are  not  all  located  in  the  same  place. 

3)  It  is  easy  to  see  many  of  the  variables  on  the  patient  monitor  (HR,  RR,  Temp),  however  that  monitor 
only  provides  the  last  10  seconds  of  data.  Longer  trends  are  not  shown  here. 

4)  Rate  of  change  of  individual  variables  is  not  easily  found 

5)  Rate  of  change  of  multiple  variables  requires  integration  in  the  clinician  or  nurses  head. 


igure  7.  Initial  Mockup  to  identify  SIRS  and  sepsis 
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In  the  initial  mockup  the  circle  (Figure  7)  was  not  actually  meant  to  be  read  (no  individual  variables  or 
numbers).  We  believed  that  showing  an  increasing  size  (and  color  and  position)  could  be  used  to  easily 
indicate  to  clinicians  that  SIRS  is  emerging.  The  clinician  or  nurse  can  then  dig  deeper  into  the  actual  numbers 
and  notice  how  those  numbers  are  changing  over  time.  The  circle  is  simply  an  alerting  mechanism.  There  are 
several  reasons  why  we  wanted  a  simple  user  interface.  In  this  case  two  variables  trend  down  while  four  trend 
up  when  a  patient  is  becoming  septic.  Secondly  this  Ul  is  new  and  therefore  we  simply  wanted  to  grab  the 
clinician’s  attention  and  direct  it  to  a  potential  event  as  evidenced  by  these  SIRS  variables  changing  in  a  bad 
direction.  It  would  then  be  up  to  the  clinician  to  dig  into  the  trends  and  order  more  tests  (if  necessary).  The 
objective  in  this  version  of  the  mockup  was  to: 

•  Present  all  6  variables  in  a  single  location 

•  Present  longer  trends  of  all  variables 

•  Show  their  individual  values  at  time  steps  of  1 5-minutes. 

•  Color  code  the  individual  values  to  indicate  how  close  they  are  to  normal  or  alarm  stages. 

As  we  explored  this  design  further  however  we  changed  it  significantly  from  the  original  mockup.  The  change 
in  design  (Figure  8)  was  prompted  by  several  concerns  with  the  original  design  including: 

1 .  The  presentation  of  the  individual  variables  and  numbers  would  not  reduce  clinician  cognitive  load. 

2.  Clinicians  would  have  to  trend  many  hours  of  data  to  determine  if  changes  were  occurring,  again 
potentially  not  positively  impacting  clinician  cognitive  load. 

3.  The  current  design  maximizes  the  effect  of  a  making  a  single  comparison  between  the  current  status 
and  past  status  and  should  be  more  effective  at  reducing  cognitive  load. 

4.  The  current  design  still  presents  all  6  variables  in  a  single  location  and  allows  the  evaluation  of  longer 
trends  of  all  variables. 
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Figure  8.  Stage  2  of  Sepsis  Mockup 
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The  design  continued  to  evolve  in  the  fourth  quarter.  We  incorporated  a  shock-index  graph.  The  shock-index 

is  a  metric  that  has  been  shown  in  studies  to  be  helpful  in  identifying  patients  at  high  risk  for  sepsis.  A  number 

of  changes  were  added  (Figure  9): 

1 .  Addition  of  the  shock  index.  The  shock  index  is  a  derived  value  that  can  be  used  to  indicate  potential 
sepsis  severity.  Completed  first  version  of  the  shock-index  graph.  Whether  the  y-axis  should  be  linear  or 
logarithmic  is  an  area  of  further  investigation. 

2.  We  reworked  the  observation  graphs  and  the  shock-index  graph  so  all  use  the  same  underlying  time-series 
graph.  This  was  necessary  to  facilitate  a  single  cursor  across  all  the  time-series  charts  (shock-index  and 
observation  graphs). 

3.  Normalization  of  the  sepsis  indicator:  The  sepsis  indicator  is  a  normalized  value.  The  normalization 
requires  some  tweaking  to  ensure  that  it  can  pick  up  increases  to  sepsis 

4.  Visualization  of  timelines  requires  additional  input  to  show  relative  increases  in  numbers  as  well  as  the 
numbers  themselves. 

5.  Changed  the  "change"  chart  to  use  a  red-gray-green  color  scheme.  The  color  selection  will  be  centralized 
so  that  the  "change"  chart,  the  patients  overview,  and  possible  others  use  the  same  "change-based" 
semantic  classifications  and  colors. 

6.  Warning  areas  for  specific  parameters  are  highlighted  in  yellow  and  red  based  on  parameter  specific 
thresholds.  The  EOF  indices  will  have  a  pop-up  showing  the  bar  graph  to  help  with  interpretation.  The 
design  allows  for  variables  to  be  added  or  removed  without  requiring  changes  to  the  interface. 
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Figure  9.  Sepsis  Visualization  Tool 
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The  PREDICT  interface  displays  meaningful  information  about  changes  in  patient  state.  Clockwise  from  top  left 
is  patient  information,  current  risk  levels  for  physiological  systems,  an  integrated  view  of  the  current  Septic 
Shock  Index,  and  the  bottom  timeline  graph  displays  trends  and  allows  drilling  into  patient  history. _ 
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Intracranial  Pressure  Management 

The  goal  and  purpose  of  the  intracranial  pressure  management  interactive  visualization  is  to  determine  an 
‘ideal’  target  for  cerebral  perfusion  pressure  that  will  maintain  adequate  perfusion  of  the  brain.  The  clinical  task 
is  to  determine  an  ‘ideal’  target  for  cerebral  perfusion  pressure  that  will  maintain  adequate  perfusion  of  the 
brain  (e.g.,  (6-8)).  Figure  10  displays  data  from  a  single  patient  and  highlights  the  complexity  of  interpreting 
patient  monitoring  data  without  visualization  as  the  relationship  between  cerebral  perfusion  pressure  (CPP) 
and  brain  oxygen  tension  is  dependent  on  end  tidal  C02  concentrations.  When  end  tidal  C02  is  below  30  the 
patient  appears  to  have  relatively  intact  cerebral  autoregulation,  which  is  to  say  that  vascular  systems  that 
regulate  cerebral  blood  flow  to  a  constant  flow  rate  are  functioning  properly  when  CPP  is  greater  than  80 
mmHg.  Note  the  locally  weighted  regression  line  is  essentially  flat  when  CPP  is  greater  than  80  mmHg,  but 
steadily  declines  below  that  pressure.  In  contrast,  when  end  tidal  C02  is  above  30  we  observe  cerebral 
autoregulation  failure  when  brain  oxygen  tension  increases  as  cerebral  perfusion  pressure  increases.  These 
relationships  are  impossible  to  determine  without  tools  to  visualize  high  resolution  physiologic  data,  which  are 
largely  not  available  at  the  bedside. 
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Interpretation  of  multimodal  data  to  identify  cerebral  perfusion  pressure  targets 


20 


00  100  140  ISO  220  20 


SO  100  140  ISO  220 


40 


30  120 


160  200  240 


40 


SO 


120  160  200  240 


C02-EX:  <=  3D 


C02-EX:  >  30 


CPP 


14 


The  figure  below  (Figure  11)  presents  the  prototype  of  the  ICP  tool. 


Figure  11.  Intracranial  Pressure  Management  Visualization 
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Real-time  visualization  demonstration 

We  have  started  to  develop  a  working  real-time  demo  that  incorporates  the  principles  of  the  three 
visualizations.  Chaos  theory  describes  the  behavior  of  attractors  in  dynamical  systems  such  as  in  human 
physiology.  In  practice  physiology  varies  around  a  physiologic  set  point  until  the  system  is  perturbed  to  a  new 
set  point.  These  ranges  can  be  identified  utilizing  financial  time  series  support  and  resistance  formulas  (Figure 
12). 

Figure  12.  Identification  of  Short-Term  Normal  Range  of  Parameters 
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Assessment  of  cerebral  autoregulation  using  the  pressure  reactivity  index  (PRx)  is  a  common  application  of 
multimodal  neuromonitoring  data.  We  have  developed  a  demonstration  real-time  web  application  to  enable 
clinicians  to  assess  cerebral  autoregulation  status  and  identify  potential  patient  state  changes  in  real-time  at 
the  bedside  (Figure  13).  We  have  uploaded  a  movie  that  demonstrates  this  functionality  (Appendix  1). 


In  addition  we  have  modified  the  basic  design  to  accommodate  multiple  data  streams  and  visualizations  from 
different  tools  being  developed.  In  Figure  14,  we  have  included  elements  from  both  the  overview  and  sepsis 
visualization  prototypes  into  a  single  tool  (Figure  14).  This  can  be  viewed  using  simulated  data  on 
http://demos.imateapot.net/neurovis/bedview.php.  This  is  hosted  on  a  low  traffic  web  server  that  can  only 
support  one  to  two  simultaneous  connections,  so  please  do  not  widely  distribute  this  link  at  this  time. 


Figure  14.  Incorporation  of  Visualization  Elements 


Cognitive  Experiment 

Phase  4  of  the  project  has  begun  and  its  objective  is  to  conduct  a  cognitive  experiment  to  test  whether  the 
visualization  improves  clinician  decision  making.  We  are  recruiting  physicians  at  three  levels  of  training  to 
participate  in  this  study.  This  includes  faculty  that  have  completed  residency  and  fellowship  in  neurocritical 
care,  as  well  as  fellow  and  resident  trainees.  To  date  we  have  consented  three  faculty  and  five  residents  to 
participate.  Seven  clinical  fellows  have  not  yet  been  approached  to  participate. 

Overview 

Physicians  are  presented  a  clinical  vignette  with  data  and  asked  to  identify  the  frequency  of  physiologic  events, 
assess  overall  patient  status,  identify  tests  and  treatments  they  would  order,  and  report  the  likelihood  that 
clinical  events  have  occurred  or  will  occur  in  the  next  24  hours. 

Design 

The  mode  of  physiologic  data  presentation  (visualization  versus  text)  and  clinical  event  type  (sepsis  versus 
other  events)  are  the  two  independent  variables.  Several  hypotheses  are  tested:  1)  The  speed  and  accuracy 
of  responses  are  better  when  physiologic  data  is  presented  in  a  visualization  versus  a  text  spreadsheet;  2) 
Clinical  events  are  more  likely  to  be  detected  in  advance  of  onset  in  the  visualization  versus  text  condition;  3) 
sepsis  detection  specifically  is  improved  when  using  data  visualization.  To  test  these  hypotheses  clinical 
vignettes  were  generated  from  actual  clinical  cases  of  3  patients  experiencing  sepsis  and  3  experiencing 
clinical  events  other  than  sepsis.  A  vignette  was  created  for  each  patient  for  the  day  the  clinical  event 
occurred,  and  for  each  of  the  2  days  leading  up  to  the  event  day.  Each  patient  day  is  presented  in  both 
visualization  and  text  form  for  a  total  of  36  (6  patients  x  3  patient  days  x  2  conditions)  vignettes. 
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The  two  most  likely  threats  to  the  internal  validity  of  the  study  were  clinicians  recognizing  that  clinical  cases 
were  presented  in  both  text  and  visualization  conditions,  and  that  clinical  events  were  likely  to  occur  on  the  3rd 
day  of  the  case.  Specifically,  if  unaltered,  the  same  history  of  present  illness  would  be  presented  six  times, 
and  the  clinical  exam  and  labs  for  each  patient  day  would  be  presented  with  physiologic  data  in  text  and 
visualization  conditions.  To  control  for  these  potential  internal  validity  threats,  we  generated  six  versions  of  the 
history  of  present  illness  and  two  versions  of  the  clinical  exam  and  labs  information.  This  was  done  in  a 
manner  that  made  it  unlikely  that  the  case  would  be  recognized  while  maintaining  the  clinical  meaning  of  the 
presentation.  In  the  instruction,  participants  are  told  that  they  would  be  presented  “a  series  of  similar  cases”  to 
reduce  questions  about  the  similarity  of  patients.  In  total,  clinicians  are  presented  36  vignettes  that  appear  to 
be  created  from  36  different  patients,  although  in  reality  come  from  6  patients.  Figure  15  shows  an  example  of 
these  modifications  for  the  same  patient  day. 

Figure  15.  Patient  Vignette  Modified  for  Visualization  and  Text  Conditions. 

Version  1 

History  of  Present  Illness: 

67  year  old  woman  with  diabetes  on  daily  baby  aspirin  who  presented  to  an  outside  hospital  with  neck  pain  and  altered  mental  status.  CT  imaging  showed  subarachnoid 
hemorrhage.  She  was  then  transferred  to  CUMC  where  repeat  imaging  showed  intraventricular  hemorrhage  (Hunt  Hess  3,  Fisher  grade  4,  modified  Fisher  grade  4),  and 
mild  hydrocephalus.  She  was  admitted  to  the  Neuro  ICU  for  further  monitoring  where  digital  subtraction  angiography  was  negative  for  aneurysms,  and  an  EVD  was 
placed. 

Post-bleed  day  9 

Physical  exam: 

■  Lungs  with  bilateral  crackles  in  bases 

*  Normal  cardiac  exam 

*  Abdomen  soft,  nontender,  non  distended,  bowel  sounds  present 

*  Mildly  erythematous  R  upper  extremity,  mild  swelling 

Neuro  exam: 

*  Asleep,  eyes  closed;  responds  to  noxious  stimulation  only,  follows  commands 

*  PERRL 

*  Localizes  with  both  UE  to  noxious  stimuli;  withdraws  with  both  LE  to  noxious  stimuli 


Version  2 

Patient  Vignette 

History  of  Present  Illness: 

64  year  old  woman  with  HTN  and  DM2  who  was  admitted  to  the  CPMC  NICU  with  Hunt  &  Hess  Grade  3,  Fisher  Grade  4,  modified  Fisher  grade  4  subarachnoid 
hemorrhage  with  early  hydrocephalus.  CT  angiography  showed  no  aneurysms  nor  evidence  of  radiographic  vasospasm,  and  no  aneurysm  was  seen  on  conventional 
angiogram.  On  arrival  to  the  NICU,  she  underwent  EVD  placement  for  hydrocephalus. 

Post-bleed  day  13 


Physical  exam: 

*  Eyes  open  spontaneously;  constant  stimulation  is  necessary  to  keep  awake 

*  Oriented  to  date;  perseverates 

•  Cranial  nerves  intact 

*  Moves  all  4  extremities  equally  and  antigravity 

Imaging: 

•  CXR  with  worsened  pulmonary  edema 

24  Hour  Labs: 

•  CSF  extracted  from  EVD:  G68  PI 27  618  WBC,  893  RBG 

The  survey  is  estimated  to  take  approximately  three  to  four  hours  to  complete.  The  survey  is  broken  down  into 
three  sessions  of  12  patient  vignettes.  Study  participants  may  also  stop  between  cases  within  a  survey. 
Balancing  was  used  to  generate  the  list  of  patient  case  days  contained  in  each  survey  (Table  1).  Lastly, 
participants  take  a  10  minute  survey  to  evaluate  the  overview  visualization  in  which  clinicians  are  presented  a 
single  visualization  and  asked  to  identify  the  three  patients  they  would  round  on  first  and  last. 

Procedure 

Physicians  are  presented  a  clinical  vignette  and  are  provided  the  history  of  present  illness,  clinical  exam 
information  and  standard  laboratory  tests  for  the  current  patient  day.  Physiologic  data  is  then  presented  in 
either  a  text  spreadsheet  similar  to  what  is  used  in  an  electronic  medical  record  or  in  the  visualization.  With  this 
information  participants  are  asked  to  identify  the  frequency  of  physiologic  events,  assess  overall  patient  status, 
identify  tests  and  treatments  they  would  order,  and  report  the  likelihood  that  clinical  events  have  occurred  or 
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will  occur  in  the  next  24  hours.  Responses  to  each  question  are  timed  allowing  speed  and  accuracy  to  be 
determined.  A  case  used  in  the  experiment  is  shown  Appendix  1-5. 

Table  1 .  Balancing  presentation  of  case  days  in  three  surveys  by  condition 


Control  Condition  Patient 

Patient  Day 

Visualization 

Text 

N01 

-2 

Survey  1 

Survey  2 

N01 

-1 

Survey  2 

Survey  3 

N01 

Event (0) 

Survey  3 

Survey  1 

N02 

-2 

Survey  2 

Survey  3 

N02 

-1 

Survey  3 

Survey  1 

N02 

Event (0) 

Survey  1 

Survey  2 

N03 

-2 

Survey  3 

Survey  1 

N03 

-1 

Survey  1 

Survey  2 

N03 

Event (0) 

Survey  2 

Survey  3 

Sepsis  Condition 

Patient 

Patient  Day 

Visualization 

Text 

SOI 

-2 

Survey  3 

Survey  1 

SOI 

-1 

Survey  1 

Survey  2 

SOI 

Event (0) 

Survey  2 

Survey  3 

S02 

-2 

Survey  2 

Survey  3 

S02 

-1 

Survey  3 

Survey  1 

S02 

Event (0) 

Survey  1 

Survey  2 

S03 

-2 

Survey  1 

Survey  2 

S03 

-1 

Survey  2 

Survey  3 

S03 

Event (0) 

Survey  3 

Survey  1 

Subjects  were  also  presented  an  EOF  composite  score  visualization  composed  of  28  patients  (See  Appendix 
6)  and  asked  to  create  a  list  the  3  patients  that  they  would  round  on  first,  and  3  patients  they  would  round  on 
last.  The  EOF  composite  score  represents  the  variation  or  change  a  patient  has  experienced  in  the  last  1,12, 
and  24  hours  in  the  cardiovascular  and  respiratory  domains.  It  also  displays  variation  within  2  undefined 
domains  and  would  represent  any  change.  Orange  represents  clinically  significant  variation.  For  example  in  the 
cardiovascular  domain  orange  represents  that  heart  rate  has  increased  while  blood  pressure  has  fallen.  Blue 
reflects  the  opposite  change  -  heart  rate  decreasing  while  blood  pressure  increases.  This  too  could  represent 
a  clinically  meaningful  change,  but  it  is  less  often  considered  life  threatening. 

Results 

EOF_  composite  score  overview  visualization 

Subjects  were  asked  to  select  3  patients  out  of  28  to  round  on  first  as  well  as  3  patients  to  round  on  last.  Four 
clinicians  (subjects)  completed  the  experiment.  The  average  time  to  complete  the  task  was  4.4  minutes. 

Three  patients  to  round  on  first: 

Subjects  reported  a  similar  rationale  for  these  choices,  which  were  the  following: 

1 .  CV  index  worsening  on  the  first  two  codes,  and  stably  poor  on  the  third  one  over  the  past  24  hrs. 

2.  All  of  these  patients  seem  to  have  worsening  of  cardiopulmonary  status  in  the  past  24  hours  in 
descending  order  of  severity. 

3.  Have  had  negative  trends  in  the  last  hour 

4.  seem  to  be  changing  the  most  in  the  last  hour 
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Table  2.  Selected  Patients  to  Round  on  First 


Patient 


Cardiovascular 
1  hr  12  hr  24  hr 


Respiratory 
1  hr  12  hr  24  hr 


Option  1 

1  hr  12  hr  24  hr 


Option  2 

1  hr  12  hr  24  hr 


62231 

19238 

34815 

49252 

51483 

71475 

85163 

2832 


Times  Selected 
by  Subjects 
3 
2 
2 
1 
1 
1 
1 
1 


A  standard  balls  and  urn  calculation  suggests  that  the  likelihood  of  2  subjects  selecting  the  same  patient  is 
0.007  compared  to  a  likelihood  of  0.12  of  any  subject  selecting  any  1  patient  from  the  list  of  28.  The 
likelihood  of  3  subjects  selecting  the  same  patient  is  0.0002. 


The  stated  rationale  for  the  patient  selection  by  subjects  stressed  cardiopulmonary  difficulty  over  respiratory 
distress,  which  is  reflected  in  the  choices  by  the  subjects.  For  these  patients  select  by  more  than  one  subject, 
two  had  the  darkest  orange  (most  change)  in  the  cardiopulmonary  index  over  the  last  hour  (62231 ,  19238)  and 
the  other  (34815)  contained  the  most  orange  in  total  over  all  three  time  periods.  Patient  2832  had  the  darkest 
blue  in  the  last  hour  of  all  patients  in  the  cardiopulmonary  index.  Patient  49252  and  51483  had  the  darkest 
orange  in  the  last  24  hours  of  all  patients  in  the  cardiopulmonary  index.  Patient  85163  had  the  darkest  orange 
in  the  12  hour  time  period  for  both  cardiopulmonary  and  respiratory  indices,  while  patient  71475  had  the  most 
orange  across  all  time  periods  in  the  respiratory  index. 

Three  patients  to  round  on  last: 

Subjects  reported  a  similar  rationale  for  these  choices,  which  were  the  following: 

1 .  CV  and  RESP  indices  look  stable  to  better;  not  sure  how  to  incorporate  the  option  1/2  indices  though 
none  of  these  are  orange  in  the  past  hr.  All  three  seem  either  stable  or  improving  (i.e.  more  blue/grey 
than  orange). 

2.  These  patients  all  appear  roughly  stable  with  less  change.  Even  though  4684  has  possibly  slight 
worsening  in  Cl,  respiratory  status  is  better  and  the  other  variables  do  not  seem  to  have  changed 
much. 

3.  These  patients  have  been  stable  to  improving 

4.  Least  change  over  24  hrs. 


Table  3.  Selected  Patients  to  Round  on  Last 


Patient 


Cardiovascular 
1  hr  12  hi  24  hi 


Respiratory 
1  hr  12  hr  24  hr 


Option  1 

1  hr  12  hr  24  hr 


Option  2 

1  hr  12  hr  24  hr 


20562 

84511 

47958 

34917 

85752 

64701 

4684 


Times  Selected 
by  Subjects 
3 
2 
2 
2 
1 
1 
1 


A  standard  balls  and  urn  calculation  suggests  that  the  likelihood  of  2  subjects  selecting  the  same  patient  is 
0.007  compared  to  a  likelihood  of  0.12  of  any  subject  selecting  any  1  patient  from  the  list  of  28.  The 
likelihood  of  3  subjects  selecting  the  same  patient  is  0.0002. 
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Subjects  again  appeared  to  choice  patients  consistent  with  their  stated  rationale.  None  of  the  subjects  had 
strong  change  (orange  or  blue)  in  the  last  hour,  and  none  had  strong  orange  at  any  point  in  the  cardiovascular 
or  respiratory  domains. 

These  data  suggest  that  color  coding  patients  allowed  clinicians  to  discriminate  stable  from  unstable  patients 
quickly.  Additional  study  is  required  to  determine  whether  detecting  physiologic  ‘change’  provides  clinicians 
essential  information  they  need  about  patients,  or  what  kind  of  metrics  might  best  discriminate  patients  that 
need  attention. 

Importantly  what  this  data  suggest  is  that  the  strategy  employed  whereby  data  is  automatically 
processed  for  clinicians  in  some  meaningful  way  and  presented  in  an  easily  understandable  format 
enabled  clinicians  to  easily  and  reliably  discriminate  patients  that  need  attention  from  those  that  do 
not.  This  strategy  appears  to  provide  users  immediate  situational  awareness  about  the  status  of  the 
patient. 

Visualization  versus  text  for_  physiologic  and_  clinical  event  detection 

Time  to  Complete.  Four  subjects  completed  the  experiment,  taking  an  average  of  4.5  hours  to  complete.  The 
average  (standard  deviation)  time  to  complete  a  case  in  the  text  condition  was  290  (122)  seconds,  which  was 
significantly  less  than  the  402  (204)  seconds  subjects  required  to  complete  the  visualization  condition  (T(69)  = 
4.636,  P<001). 

The  text  condition  was  designed  to  be  similar  to  clinical  conditions  in  the  ICU.  The  rate  of  physiologic 
data  was  one  value  every  hour  whereas  for  the  visualization  condition  data  was  presented  once  every 
minute.  Our  data  suggest  that  presenting  more  raw  data,  even  in  visual  form,  increases  the  time  it 
takes  clinicians  to  process  that  data. 

Physiologic  Event  Detection.  Subjects  were  asked  to  identify  the  number  of  physiologic  events  that  occurred 
during  the  last  24  hours  and  also  determine  whether  the  patient  has  gotten  better  or  worse  in  that  time.  The 
average  weighted  Cohen’s  Kappa  of  the  four  subjects  was  not  significantly  different  for  any  event.  This  data 
suggests  that  clinician  judgments  regarding  the  occurrence  of  physiologic  events  did  not  differ 
depending  on  whether  data  was  presented  in  text  or  visual  form  (Table  4). 

Clinical  Event  Detection.  Subjects  were  asked  what  clinical  diagnoses  they  believed  were  present  in  the  last  24 
hours  and  what  events  would  likely  occur  in  the  next  24  hours.  Data  was  analyzed  with  the  nonparametric 
Wilcoxon  rank  sum  test  with  continuity  correction  using  a  p-value  threshold  of  0.01  to  determine  statistical 
significance.  There  was  no  evidence  to  suggest  that  raw  data  presented  in  visual  form  provide  clinicians 
additional  situational  awareness  or  increased  certainty  regarding  the  presentation  of  clinical 
complications  (Table  5). 


Table  4:  Clinician  Agreement  on  Occurrence  of  Physiologic  Events 


Clinical  Events 

Average  Weighted  Cohen's  Kappa 

Text  Condition  Visual  Condition 

T-Score 

P-Value 

Overall 

0.25 

0.27 

-0.38 

0.71 

Bradycardia 

0.30 

0.28 

0.14 

0.89 

Fever 

0.32 

0.34 

-0.06 

0.95 

Hypothermia 

0.34 

0.36 

-0.07 

0.94 

Hyperventilation 

0.10 

0.13 

-0.15 

0.88 

Hypoxia 

0.23 

0.22 

0.06 

0.96 

Hypertension 

-0.02 

0.21 

-1.2 

0.26 

Hypotension 

0.35 

0.40 

-0.19 

0.85 

ICP  elevation 

0.34 

0.24 

0.46 

0.65 
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Table  5:  Clinician  Certainty  of  Clinical  Complications 


Clinical  Events 

Control  Cases 

Event  Cases 

Wilcox  W 

P-Value 

Sepsis  (Last  24  hours) 

Text 

2  (2,  3) 

2  (2,  3) 

515 

0.41 

Visualization 

2  (2,  3) 

2.25(2,3) 

492 

0.13 

Text  -  Visual 

0  (-0.5, 1) 

0  (-1,  0) 

623 

0.41 

Sepsis  (Next  24  hours) 

Text 

3  (2,  3) 

3  (2,  3.75) 

410 

0.03 

Visualization 

3  (2,  3) 

3  (2,  3.5) 

539 

0.36 

Text  -  Visual 

0  (-1,  0) 

0  (-0.5,  0.25) 

479 

0.27 

Respiratory  Failure 
(Last  24  hours) 

Text 

3  (2,  4) 

3  (2,  4) 

527 

0.77 

Visualization 

3  (2,  4) 

3  (2,  4.5) 

478 

0.43 

Text  -  Visual 

0  (-1,  1) 

0  (-1,  0) 

559 

0.38 

ICP  Crisis  (Next  24 
hours) 

Text 

3  (2,  4) 

3  (1,  4) 

551 

0.55 

Visualization 

3  (2.5,  4) 

3.5  (3,  4) 

451 

0.24 

Text  -  Visual 

0  (-1,  1) 

0  (-1,  0) 

577 

0.26 

Ratings:  0  =  'Impossible'; 

1  =  'Improbable'; 

2  =  '50/50';  3  = 

'Probable';  4  = 

'Certain' 

Discussion 

Although  the  conducted  study  is  limited  by  the  small  number  of  clinicians  tested,  it  does  appear  to  illuminate  a 
number  of  issues.  Contrary  to  our  hypothesis,  it  our  clinicians  took  longer  to  process  and  make  decisions 
about  patients  in  the  visualization  conduction.  Raw  data  was  presented  at  a  higher  frequency  in  the 
visualization  (1  data  point  /  minute)  over  text  (1  data  point  / 15  minutes)  condition.  It  is  possible  that  clinicians 
are  more  practiced  at  synthesizing  text  information,  however  it  seems  most  likely  that  have  more  data 
increased  the  time  clinicians  needed  to  determine  how  long  a  patient  experienced  different  physiologic  events. 
This  interpretation  is  consistent  with  findings  from  other  studies.  (9-12) 

There  was  also  no  evidence  to  suggest  that  simply  providing  raw  data  in  a  visual  form  improved  the 
identification  of  physiologic  events,  or  improved  the  clinical  recognition  of  clinical  complications,  that  would 
warrant  providing  clinicians  higher  resolution  raw  data.  We  did  not  find  any  meaningful  differences  between 
the  text  and  visualization  conditions  to  suggest  that  the  absence  of  findings  is  due  to  insufficient  statistical 
power.  Together  this  suggests  that  presenting  more  data  increases  clinicians’  cognitive  load  even  if  data  is 
presented  visually,  and  that  simply  presenting  such  data  in  visual  format  alone  does  not  reduce  clinician 
cognitive  load.  In  contrast  when  synthesized  data  was  presented  to  clinicians  (overview  visualization)  clinicians 
were  able  to  use  reliably  to  prioritize  patients  that  needed  attention  and  identify  patients  that  were  stable. 

This  finding  is  in  alignment  with  the  idea  of  automatic  processing  of  information  for  clinicians  can  streamline 
clinical  decision  making.  Data  should  be  processed  for  clinicians  automatically  and  results  presented  in 
a  simple  display.  For  example,  instead  of  clinicians  determining  the  time  patients  were  tachycardic  this 
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information  should  be  calculated  automatically.  Clinicians  would  simply  be  shown  that  the  patient  had  been 
tachycardic  for  5  hours  in  the  last  24  hours. 

It  has  been  shown  that  taking  information  from  many  screens  to  one  screen  improves  clinician  decision  making 
(1 1 )  and  it  is  known  that  how  information  is  presented  influences  decisions  (13).  Data  visualization  remains 
important.  However  our  (limited)  data  do  not  suggest  that  providing  visual  data  to  clinicians  will  help  them 
detect  subtle  changes  in  physiology  that  may  indicate  complex  events  like  sepsis.  A  key  distinction  must  be 
made  between  visual  data  representation  from  decision  support.  There  are  many  ways  in  which  data  can 
be  visualized  to  represent  knowledge,  however  representing  data  visually  still  requires  the  user  to  extract 
meaning  from  it.  Situational  awareness  will  be  most  enhanced  by  automatically  extracting  information  from 
data  and  converting  that  data  into  information  that  clinicians  can  use  to  make  a  decision  (e.g.,  risk  of  a 
complication,  frequency  of  physiologic  events). 

Meaningful  ways  to  accomplish  data  reduction  is  the  biggest  barrier  to  improving  clinical  decision 
making.  We  recommend  that  future  studies  focus  on  methods  to  reduce  the  number  of  data  points  clinicians 
must  process  to  obtain  situational  awareness  about  patients.  The  current  study  explored  one  possible  way  to 
discriminate  stable  from  unstable  patients  using  linear  projections  of  multivariate  time  series  coupled  with  a 
simple  color-coded  scheme.  It  is  unknown  if  this  particular  method  is  a  valuable  to  clinicians,  but  our  data 
clearly  suggest  that  clinicians  were  able  to  quickly  and  reliably  identify  patients  that  were  identified  by  color.  A 
cautionary  note  is  that  while  color  coding  based  on  good  underlying  information  is  valuable,  color  coding  based 
on  a  poor  strategy  (e.g.,  acuity)  will  be  unhelpful  or  potentially  dangerous. 

Key  Research  Accomplishments 

•  Devised  methods  to  both  filter  and  interpolate  physiological  data. 

•  Created  physiologic  indices  for  clinically  meaningful  variation  among  several  parameters  using  a 
version  of  empirical  orthogonal  functions.  This  method  is  generalizable  to  track  any  combination  of 
parameters  that  are  deemed  clinically  important  to  track. 

•  Applied  the  above  physiologic  indices  to  create  a  patient  overview  visualization  to  help  clinicians 
identify  patients  that  are  changing  and  inserted  these  indices  into  the  sepsis  specific  decision  support 
visualization. 

•  Created  a  sepsis  identification  visualization  tool  to  help  clinicians  identify  patients  headed  for  septic 
shock. 

•  Generated  a  visualization  tool  to  help  clinicians  assess  intracranial  pressure  and  generate  patient 
specific  physiologic  set-points  for  goal  directed  therapy. 

•  Demonstrate  a  real-time  data  feed  to  a  visualization  tool  that  contains  elements  from  each  of  the  three 
prototype  visualizations. 

Future  Plans 

Our  cognitive  work  analysis  identified  a  need  for  a  patient  visualization  that  provides  an  overview  of  all  critical 
systems  for  the  patient,  including  labs  and  medications.  Our  cognitive  experiment  suggests  that  viable 
methods  to  provide  clinicians  actionable  information  while  reducing  the  volume  of  data  to  synthesize  is 
fundamental  to  this  effort.  The  workflow  progression  would  be  to  use  the  overview  visualization  developed  in 
this  proposal  to  identify  which  patient  needs  to  be  evaluated.  The  next  visualization  would  then  provide  a 
broad  overview  for  an  individual  patient,  including  elements  that  have  been  developed  in  this  proposal,  as  well 
as  additional  elements  that  help  the  clinician  identify  clinical  events  and  alarm  status,  the  physiologic  state  of 
organ  systems  including  labs,  and  current  medications. 

We  have  begun  to  create  multiple  mockups  for  other  interactive  visual  support  tools  to  help  clinicians  with  other 
cognitive  and  collaborative  challenges  (See  uploaded  powerpoint  presentation).  We  have  also  connected  our 
web-based  tool  to  a  real-time  data  feed,  but  there  are  limitations  to  this  approach  in  terms  of  how  much 
information  a  web  browser  can  hold  without  experiencing  slowdown.  There  is  a  need  to  study  more  intensively 
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what  the  right  combination  of  static  and  live  data  should  be  used  to  support  visualization  applications.  These 
ideas  are  worthy  of  further  study  and  as  such  our  proposed  plans  for  next  steps  include: 

Aim  #1 :  Development  methods  of  data  reduction  -  conduct  observational  studies  that  evaluate  the  clinical 
meaning  of  different  mulitivariate  linear  projections  of  physiology  processes  developed  in  this  proposal. 
Determine  what  information  clinicians  use  to  make  specific  decisions,  and  devise  methods  to  generate  this 
information,  minimizing  or  eliminating  the  need  for  clinicians  to  synthesize  raw  data  into  actionable  information. 

Aim  #2:  Refine  the  current  prototype  -  Adapt  the  current  prototype  to  provide  synthesized  data  to  clinicians 
utilizing  ecological  interface  design  principles.  Based  on  our  experiment  results  the  first  step  would  be  to 
further  refine  the  designs  and  add  a  new  patient-specific  summary  page.  We  have  included  a  powerpoint 
presentation  (Appendix  A)  with  mockups  that  expand  on  many  of  these  ideas. 

Aim  #3:  Transition  the  current  prototype  to  real-time  system  -  This  would  involve  data  model  development 
to  ensure  the  appropriate  data  syntax  and  semantics  are  represented  for  the  real-time  PREDICT  system.  Then 
the  IMEDS  middleware  would  be  implemented  to  provide  the  appropriate  services  while  the  IMEDS  API  is 
developed  to  support  the  PREDICT  system  requirements  and  ICE  compliance. 

Aim  #4:  Deploy  and  evaluate  PREDICT  system  at  three  sites  (Columbia,  Emory,  University  of  Maryland) 
to  determine  impact  on  results  of  patient  outcomes  and  paths  to  those  outcomes  -  An  evaluation 
including  a  structured  survey  of  clinicians’  experience  in  ease  of  data  acquisition,  quality  of  decision-making, 
and  time  saved  (or  wasted),  and  patients’  clinical  outcomes. 

Conclusions 

In  the  intensive  care  unit  environment  clinicians  are  under  a  constant  state  of  information  overload.  Patients  in 
need  to  medical  attention  are  not  identified  in  a  time  to  treat  them  beneficially,  and  mismanagement  from  a  lack 
of  understanding  of  patient  occurs  too  frequently.  These  problems  lead  to  lower  quality  patient  care  resulting 
in  more  secondary  complications,  less  than  optimal  outcomes  for  patients,  and  longer  ICU  length  of  stay 
increasing  healthcare  costs.  Results  from  our  cognitive  work  analysis  confirmed  the  need  for  improvements  in 
patient  information  presentation  to  facilitate  clinical  decision  making.  We  identified  the  need  for  a  patient 
overview  screen  that  contains  data  from  bedside  monitors,  electronic  health  record,  and  laboratory  systems. 
This  type  of  visualization  was  not  within  the  original  scope  of  work  and  remains  an  important  unmet  need.  The 
three  visualization  prototypes  that  we  did  develop  are  meant  to  help  clinicians  on  a  variety  of  important  clinical 
tasks  including:  1)  prioritizing  patients  most  in  need  of  medical  attention,  2)  identifying  patients  at  high  risk  for 
sepsis,  and  3)  managing  intracranial  pressure  crises  and  identifying  patient  specific  physiologic  targets.  A 
cognitive  experiment  will  be  conducted  in  the  next  period  to  determine  what  works  and  what  does  not  work 
regarding  our  visualization  designs;  this  will  allow  firmer  conclusions  to  be  drawn  and  will  support  the  next 
phase  of  development  for  medical  data  visualization  and  decision  support  in  the  intensive  care  unit 
environment. 

Several  steps  need  to  be  completed  to  move  towards  a  deployable  clinical  decision  support  tool.  A  new  phase 
of  visualization  development  is  needed  informed  by  the  findings  of  the  cognitive  experiment.  The  visualization 
tools  need  to  be  optimized  for  real-time  deployment,  either  fed  by  a  real-time  data  stream  as  we  have 
demonstrated  here  or  fed  from  a  real-time  file  structure.  Deploying  and  evaluating  the  visualization  tools  to 
multiple  sites  is  a  crucial  step  in  this  development.  This  will  provide  a  framework  for  general  deployment  and 
provide  an  opportunity  to  evaluate  these  designs  clinically  to  determine  if  in  fact  decision  support  tools  improve 
clinician  decision  making  resulting  in  less  secondary  complications,  shorter  ICU  length  of  stay,  and 
improvements  in  long-term  outcome. 
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What  opportunities  for  training  and  profession  development  has  the 
project  provided? 

Nothing  to  report. 


How  were  the  results  disseminated  to  the  communities  of  interest? 

Findings  from  our  cognitive  work  analysis  were  incorporated  into  a  book  chapter  regarding  data  visualization  in 
neurocritical  care.  Our  visualization  tools  have  been  presented  to  thought  leaders  regarding  clinical  informatics 
in  the  intensive  care  community  in  the  form  of  an  abstract  and  two  invited  talks  at  an  international  conference 
and  symposium.  The  results  of  the  cognitive  experiment  were  recently  presented  at  an  international 
symposium  on  applications  of  streaming  analytics  in  critical  care.  Java  code  for  the  real-time  visualization  has 
been  shared  with  investigators  at  Harvard  University. 

•  Schmidt,  J.  M.,  Irvine,  J.  M.,  &  Miller,  S.  (in  press).  Data  Visualization.  In  M.  DeGeorgia  &  K.  Loparo 
(Eds.),  Neurocritical  Care  Informatics:  Translating  Data  into  Bedside  Action :  Springer. 

•  Schmidt,  J.  M.,  Doerner,  J.,  Sow,  D.,  Perotte,  A.,  &  Mayer,  S.  Bedside  Application  for  the  Assessment 
of  Cerebral  Autoregulation  and  Patient  State  Changes.  Neurocritical  Care,  2013.  19,  S205. 

•  Schmidt,  J.  M.  (2013,  November).  Streaming  Analytics  Case  Study  in  Columbia  University  Neuro-ICU 
Paper  presented  at  the  EME  Streaming  Analytics  Symposium,  Chicago,  Illinois. 

•  Schmidt,  J.  M.  (2014,  August).  What  am  I  thinking  about?  International  Conference  on  Complexity  of 
Acute  Illness.  Charlottesville,  Virginia. 

•  Schmidt,  J.  M.  (2015,  September).  Results  from  Data  Visualization  Cognitive  Experiment.  Paper 
presented  at  the  EME  Streaming  Analytics  Symposium,  Chicago,  Illinois. 
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What  do  you  plan  to  do  during  the  next  reporting  period  to  accomplish  the 
goals? 

Nothing  to  report. 

IMPACT 

What  was  the  impact  on  the  development  of  the  principal  discipline(s)  of  the  project? 

This  project  was  impactful  primarily  in  three  ways.  First  we  demonstrated  the  application  of  a  method  to 
reduce  multiple  related  high  resolution  physiologic  parameters  to  a  single  that  can  be  tracked  over  time,  used 
to  alert  of  clinical  staff,  and  used  as  the  basis  of  a  clinical  decision  support  tool  that  helps  clinicians  identify 
patients  that  are  either  unstable  or  have  changed  in  status  over  a  specific  time  period.  Most  importantly,  this 
methods  overlays  clinical  knowledge  to  the  data  such  that  physiologic  patterns  that  are  known  to  be  dangerous 
to  the  patient  can  be  tracked.  Second,  we  had  multiple  opportunities  to  convey  to  the  larger  research 
community  that  it  is  insufficient  to  improve  clinical  decision  making  by  simply  presenting  raw  data  in  visual 
form.  The  field  is  working  to  address  this  in  two  ways.  One  is  to  have  dedicated  e-icu  staff  track  patient  data 
24/7  looking  for  signs  of  physiologic  derangement  or  imminent  clinical  events.  When  potential  problems  are 
observed  they  call  the  clinicians  treating  that  patients  and  let  them  know  to  go  check  their  patient  and  give  an 
indication  as  to  their  concern.  This  approach  allows  nurses  in  particular  to  complete  their  duties  without  having 
to  constantly  look  at  the  data  for  problems,  which  is  unrealistic  and  compounded  by  alarm  fatigue.  The  best 
example  of  this  are  initiatives  at  Emory  University  led  by  Tim  Buchman,  MD.  The  second  approach  has  been  to 
continue  to  develop  data  reduction  methods  that  extract  clinically  important  information  to  providers.  Instead  of 
providers  looking  at  data  and  deducing  for  themselves  that  there  may  be  a  problem,  computer  algorithms 
automatically  process  large  volumes  of  data  and  providers  clinicians  a  risk  assessment  for  different  clinical 
events.  This  information  can  be  more  easily  incorporated  into  clinical  workflow.  Lastly,  code  to  generate  the 
visualization  tools  tested  in  this  project  are  available  to  the  research  community  and  has  already  been  shared 
with  investigators  at  Harvard  University. 

What  was  the  impact  on  other  disciplines? 

Nothing  to  report. 

What  was  the  impact  on  technology  transfer? 

Nothing  to  report. 

What  was  the  impact  on  society  beyond  science  and  technology? 

Nothing  to  report. 

CHANGES/PROBLEMS 

Changes  in  approach  and  reason  for  change. 

We  deployed  a  back-end  MongoDB  database  and  was  able  to  demonstrate  depositing  data  into  it.  However, 
connecting  the  visualization  tool  directly  to  this  data  source  proved  to  not  be  realistic  given  the  time  constraints 
and  need  for  experimenter  control.  Instead  we  created  static  files  that  were  read  directly  for  the  purpose  of  the 
experiment.  This  did  not  impact  achieving  the  primary  goals  of  the  project  and  we  instead  demonstrated  the 
capacity  for  the  visualization  tools  to  connect  to  a  real-time  data  stream  instead. 
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Actual  or  anticipated  problems  or  delays  and  actions  or  plans  to  resolve 
them. 

Nothing  to  report. 

Changes  that  had  a  significant  impact  on  expenditures. 

Nothing  to  report. 


Significant  changes  in  the  use  or  care  of  human  subjects,  vertebrate 
animals,  biohazards,  and/or  select  agents. 

Nothing  to  report. 

PRODUCTS 

Publications,  conference  papers,  and  presentations 

Journal  publications 

1.  Schmidt,  J.  M.,  Miller,  S.,  Perotte,  A.,  Kummer,  B.,  Doerner,  J.,  Meyers,  E.,  Mayer,  S.  A.  Patient  Data 
Visualization  not  Sufficient  to  Improve  Situational  Awareness,  (in  preparation). 

Book  or  other  non-periodical,  one-time  publications 

1.  Schmidt,  J.  M.,  Irvine,  J.  M.,  &  Miller,  S.  (in  press).  Data  Visualization.  In  M.  DeGeorgia  &  K.  Loparo 
(Eds.),  Neurocritical  Care  Informatics:  Translating  Data  into  Bedside  Action :  Springer. 

Other  publications,  conference  papers,  and  presentations 

1 .  Schmidt,  J.  M.,  Doerner,  J.,  Sow,  D.,  Perotte,  A.,  &  Mayer,  S.  Bedside  Application  for  the  Assessment 
of  Cerebral  Autoregulation  and  Patient  State  Changes.  Neurocritical  Care,  2013.  19,  S205. 

2.  Schmidt,  J.  M.  (2013,  November).  Streaming  Analytics  Case  Study  in  Columbia  University  Neuro-ICU 
Paper  presented  at  the  EME  Streaming  Analytics  Symposium,  Chicago,  Illinois. 

3.  Schmidt,  J.  M.  (2014,  August).  What  am  I  thinking  about?  International  Conference  on  Complexity  of 
Acute  Illness.  Charlottesville,  Virginia. 

4.  Schmidt,  J.  M.  (2015,  September).  Results  from  Data  Visualization  Cognitive  Experiment.  Paper 
presented  at  the  EME  Streaming  Analytics  Symposium,  Chicago,  Illinois. 

Website(s)  or  other  Internet  site(s) 

Nothing  to  report 


Technology  or  techniques 

Nothing  to  report 
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Inventions,  patient  applications,  and/or  licenses 

Nothing  to  report 


Other  products 

1 .  Created  a  sepsis  identification  visualization  tool  to  help  clinicians  identify  patients  headed  for  septic 
shock. 

2.  Generated  a  visualization  tool  to  help  clinicians  assess  intracranial  pressure  and  generate  patient 
specific  physiologic  set-points  for  goal  directed  therapy. 

3.  Created  a  visualization  that  can  accept  a  real-time  data  feed. 
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APPENDICES 

1.  Initial  patient  vignette 

Patient  Case 


Patient  Vignette 

History  of  Present  Illness: 

64  year  old  woman  with  HTN  and  DM2  who  was  admitted  to  the  CPMC  NICU  with  Hunt  &  Hess  Grade  3,  Fisher  Grade  4,  modified  Fisher  grade  4  subarachnoid 
hemorrhage  with  early  hydrocephalus.  CT  angiography  showed  no  aneurysms  nor  evidence  of  radiographic  vasospasm,  and  no  aneurysm  was  seen  on  conventional 
angiogram.  On  arrival  to  the  NICU,  she  underwent  EVD  placement  for  hydrocephalus. 

Post-bleed  day  13 


Physical  exam: 

•  Eyes  open  spontaneously;  constant  stimulation  is  necessary  to  keep  awake 

•  Oriented  to  date;  perseverates 

•  Cranial  nerves  intact 

•  Moves  all  4 extremities  equally  and  antigravity 

Imaging: 

•  CXRwith  worsened  pulmonary  edema 

24  Hour  Labs: 

•  CSF  extracted  from  EVD:  G68  P127  618  WBC,  893  RBC 


136 

101 

15 

3.3 

23 

1.1 

41.  Please  copy  the  following  text  into  firefox  to  view  the  patient  data.  To  confirm  that  you  have  the  correct  data  for  this  patient,  please  enter  the  last  observed  heart  rate 
measurement  for  this  period.  The  other  questions  will  appear  once  the  correct  value  is  entered.  If  you  have  any  problem  please  ask  for  help. 

file:///O:/NICU/Shared%20Projects/Michael%20Schmidt/Experiment/PREDICT%20-%20vl0/PREDICT/index.html#/patients/90331* 


Next 
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2.  Patient  data  presented  in  visualization 


Patient 


Name  90331 

Id  90331 


EOF 


1  hr  12  hr  24  hi 


Cardiovascular  | 
Respiratory  | 
Option  1 1 
Option  2 1 


About  Sponsors  Contact 


Change 

WBC  Shock  Index 
ICP  cv 


:02-EX 

*  ^ 

TEMP  | 

SP02 


resp 
eofl 
eof2 


RR  -  ^  HR 

MAP  SBP 


Graphs  Text 


Observations 


Shock 

Index 


Current 

0.28 


0.35 


CV 


-1.95 


-1.3 

24  hr 


resp 


Current 

-0.93 


-0  76 

24  hr 


eofl 


V/w  \±A 


r5 


0.78 

24  hr 


eof2 


Current 

-0.27 


-0.25 

24  hr 


HR 

ppm 


Current 

-  52 


57 


SBP 

mmHG 


185.5 


163.5 

24  hr 


MAP 


106 


121 

24  hr 
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3.  Patient  data  presented  in  text  spreadsheet 


Time 

20:00 

21:00 

22:00 

23:00 

0:00 

1:00 

2:00 

3:00 

4:00 

5:00 

6:00 

7:00 

8:00 

9:00 

10:00 

Shock  Inc 

0.4 

0.53 

0.37 

0.32 

0.33 

0.52 

0.47 

0.47 

0.48 

0.47 

0.57 

0.46 

cv 

-1.14 

-0.4 

-1 .96 

-2.01 

-1 .86 

-0.08 

-0.35 

-0.41 

-0.94 

-0.23 

-0.5 

0.23 

-0.36 

resp 

1.35 

1.03 

2.62 

-0.49 

-0.09 

-0.33 

-0.48 

-0.13 

-0.99 

-1 .09 

-0.99 

-0.12 

-1 .41 

eofl 

0.91 

0.47 

2.29 

1.78 

1 .67 

-0.35 

-0.12 

0.06 

0.56 

-0.41 

0.06 

-0.91 

-0.35 

eof2 

-4.04 

-2.78 

-5 

-0.06 

-0.13 

-0.06 

-0.08 

-0.1 

0.09 

-0.05 

0.07 

-2.14 

-0.01 

HR 

66 

81 

79 

64 

64 

65.5 

63 

65 

65 

59 

65 

60 

58 

SBP 

167 

153 

215 

197.5 

194 

125 

133 

139 

123 

139 

106 

125 

MAP 

93 

91 

146 

144 

140 

73.5 

77 

79 

73 

85 

64 

75 

RR 

21 

18 

28 

20 

24 

22 

21 

23 

15 

15 

15 

14 

12 

SP02 

96 

97 

95 

100 

100 

100 

100 

100 

100 

100 

100 

98 

100 

TEMP 

C02-EX 

ICP 

17 

6 

11 

9 

15 

23 

12 

8 

17 

17 

15 

WBC 

7.2 
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4.  Survey  questions:  identify  events,  patient  status,  tests,  and  orders 

42.  Which  events  occurred  during  the  time  period  shown?  * 


Never  occurred 

<  1  hour 

1-  3  hours 

3- 12  hours 

>  12  hours 

U  n  a  v  a  i  1  a  bl  e/U  n  k  now  n 

Tachycardia 

o 

o 

o 

o 

o 

o 

Bradycardia 

o 

o 

o 

o 

o 

o 

Fever 

o 

o 

o 

o 

o 

o 

Hypothermia 

o 

o 

o 

o 

o 

o 

Hyperventffation 

o 

o 

o 

o 

o 

o 

Hypoxia 

o 

o 

o 

o 

o 

o 

Hypertension 

o 

o 

o 

o 

o 

o 

Hypotension 

o 

o 

o 

o 

o 

o 

ICP  efevation 

o 

o 

o 

o 

o 

o 

43.  How  has  the  patient  CHANGED  during  the  time  period  shown  (i.e.  has  the  physiology  gotten  better  or  worse  during  the  day)?* 

Much  Improved  Improved  No  Change  Worse  Much  Worse  Unavailable/Unknown 

Patient  Status  O  O  O  O  O  O 

44.  Whfch  tests  (if  any)  would  you  order  for  this  patient  based  on  the  information  provided?  * 


45.  Are  there  any  therapeutics  that  you  would  order? 


5.  Likelihood  of  clinical  events  in  the  last  24  hours  and  next  24  hours 

46.  Given  the  information  available  to  you,  what  is  the  likelihood  that  this  patient  has  experienced  any  of  these  conditions  within  the  LAST  24  hours?  * 


Impossible 

Improbable 

50/50 

Probable 

Certain 

Unknown 

Cerebral  Vasospasm 

o 

o 

o 

o 

o 

o 

ICP  Crisis 

o 

o 

o 

o 

o 

o 

Respiratory  Failure 

o 

o 

o 

o 

o 

o 

Sepsis 

o 

o 

o 

o 

o 

o 

Shock 

o 

o 

o 

o 

o 

o 

Ventilator  Associated  Pneumonia 

o 

o 

o 

o 

o 

o 

Ventriculitis 

o 

o 

o 

o 

o 

o 

47.  Given  the  information  available  to  you,  what  is  the  likelihood  that  this  patient  W/ILL  €Xp€rf6nC6  an  y  of  these  conditions  in  the  NEXT 24  hours?* 


Impossible 

Improbable 

50/50 

Probable 

Certain 

Unknown 

Cerebral  Vasospasm 

o 

o 

o 

o 

o 

o 

ICP  Crisis 

o 

o 

o 

o 

o 

o 

Respiratory  Failure 

o 

o 

o 

o 

o 

o 

Sepsis 

o 

o 

o 

o 

o 

o 

Shock 

o 

o 

o 

o 

o 

o 

Ventilator  Associated  Pneumonia 

o 

o 

o 

o 

o 

o 

Ventriculitis 

o 

o 

o 

o 

o 

o 

Next 
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6.  Presented  EOF  composite  score  overview  visualization 


Patients 


Overview 

p.  ,,  ,  Cardiovascular  Respiratory  Option  1  Options 

Kaiieni  1  hr  12hr  24hr  1  hr  12hr  24hr  1  hr  12hr  24hr  1  hr  12hr  24hr 

46694 
27762 
7261 
89158 
29787 
4684 
64701 
71475 
85752 
47958 
85163 
17822 
34815 
37695 
8881 
51483 
46565 
58534 
58377 
84511 
56902 
78934 
49252 
2832 
62231 
19238 
20562 
34917 
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